Paper 3: Selecting rapid review methods for complex questions related to health policy and system issues

Approaches for rapid reviews that focus on streamlining systematic review methods are not always suitable for exploring complex policy questions, as developing and testing theories to explain these complexities requires configuring diverse qualitative, quantitative, and mixed methods studies. Our objective was therefore to provide a guide to selecting approaches for rapidly (i.e., within days to months) addressing complex questions related to health policy and system issues. We provide a two-stage, transdisciplinary collaborative process to select a rapid review approach to address complex policy questions, which consists of scoping the breadth and depth of the literature and then selecting an optimal approach to synthesis. The first stage (scoping the literature) begins with a discussion with the stakeholders requesting evidence to identify and refine the question for the review, which is then used to conduct preliminary searches and conceptually map the documents identified. In the second stage (selection of an optimal approach), further stakeholder consultation is required to refine and tailor the question and approach to identifying relevant documents to include. The approach to synthesizing the included documents is then guided by the final question, the breadth and depth of the literature, and the time available and can include a static or evolving conceptual framework to code and analyze a range of evidence. For areas already covered extensively by existing systematic reviews, the focus can be on summarizing and integrating the review findings, resynthesizing the primary studies, or updating the search and reanalyzing one or more of the systematic reviews. The choice of approaches for conducting rapid reviews is intertwined with decisions about how to manage projects, the amount of work to be done, and the knowledge already available, and our guide offers support to help make these strategic decisions

SUPPORT Tools for evidence-informed health Policymaking (STP) 4: Using research evidence to clarify a problem

This article is part of a series written for people responsible for making decisions about health policies and programmes and for those who support these decision makers

Within-group behavioral variation promotes biased task performance and the emergence of a defensive caste in a social spider

The social spider Anelosimus studiosus exhibits a behavioral polymorphism where colony members express either a passive, tolerant behavioral tendency (social) or an aggressive, intolerant behavioral tendency (asocial). Here we test whether asocial individuals act as colony defenders by deflecting the suite of foreign (i.e., heterospecific) spider species that commonly exploit multi-female colonies. We (1) determined whether the phenotypic composition of colonies is associated with foreign spider abundance, (2) tested whether heterospecific spider abundance and diversity affect colony survival in the field, and (3) performed staged encounters between groups of A. studiosus and their colony-level predator Agelenopsis emertoni (A. emertoni)to determine whether asocial females exhibit more defensive behavior. We found that larger colonies harbor more foreign spiders, and the number of asocial colony members was negatively associated with foreign spider abundance. Additionally, colony persistence was negatively associated with the abundance and diversity of foreign spiders within colonies. In encounters with a colony-level predator, asocial females were more likely to exhibit escalatory behavior, and this might explain the negative association between the frequency of asocial females and the presence of foreign spider associates. Together, our results indicate that foreign spiders are detrimental to colony survival, and that asocial females play a defensive role in multi-female colonies

Effects of an evidence service on health-system policy makers' use of research evidence: A protocol for a randomised controlled trial

<p>Abstract</p> <p>Background</p> <p>Health-system policy makers need timely access to synthesised research evidence to inform the policy-making process. No efforts to address this need have been evaluated using an experimental quantitative design. We developed an evidence service that draws inputs from Health Systems Evidence, which is a database of policy-relevant systematic reviews. The reviews have been (a) categorised by topic and type of review; (b) coded by the last year searches for studies were conducted and by the countries in which included studies were conducted; (c) rated for quality; and (d) linked to available user-friendly summaries, scientific abstracts, and full-text reports. Our goal is to evaluate whether a "full-serve" evidence service increases the use of synthesized research evidence by policy analysts and advisors in the Ontario Ministry of Health and Long-Term Care (MOHLTC) as compared to a "self-serve" evidence service.</p> <p>Methods/design</p> <p>We will conduct a two-arm randomized controlled trial (RCT), along with a follow-up qualitative process study in order to explore the findings in greater depth. For the RCT, all policy analysts and policy advisors (n = 168) in a single division of the MOHLTC will be invited to participate. Using a stratified randomized design, participants will be randomized to receive either the "full-serve" evidence service (database access, monthly e-mail alerts, and full-text article availability) or the "self-serve" evidence service (database access only). The trial duration will be ten months (two-month baseline period, six-month intervention period, and two month cross-over period). The primary outcome will be the mean number of site visits/month/user between baseline and the end of the intervention period. The secondary outcome will be participants' intention to use research evidence. For the qualitative study, 15 participants from each trial arm (n = 30) will be purposively sampled. One-on-one semi-structured interviews will be conducted by telephone on their views about and their experiences with the evidence service they received, how helpful it was in their work, why it was helpful (or not helpful), what aspects were most and least helpful and why, and recommendations for next steps.</p> <p>Discussion</p> <p>To our knowledge, this will be the first RCT to evaluate the effects of an evidence service specifically designed to support health-system policy makers in finding and using research evidence.</p> <p>Trial registration</p> <p>ClinicalTrials.gov: <a href="http://www.clinicaltrials.gov/ct2/show/NCT01307228">NCT01307228</a></p

SUPPORT Tools for evidence-informed health Policymaking (STP) 5: Using research evidence to frame options to address a problem

This article is part of a series written for people responsible for making decisions about health policies and programmes and for those who support these decision makers

Does dissemination extend beyond publication: a survey of a cross section of public funded research in the UK

Background: In the UK, most funding bodies now expect a commitment or effort on the part of grant holders to disseminate the findings of their research. The emphasis is on ensuring that publicly funded research is made available, can be used to support decision making, and ultimately improve the quality and delivery of healthcare provided. In this study, we aimed to describe the dissemination practices and impacts of applied and public health researchers working across the UK.Methods: We conducted a survey of 485 UK-based principal investigators of publicly funded applied and public health research. Participants were contacted by email and invited to complete an online questionnaire via an embedded URL. Gift vouchers were given to all participants who completed the questionnaire. Four reminder emails were sent out to non-respondents at one, two, three, and four weeks; a fifth postal reminder was also undertaken.Results: A total of 243/485 (50%) questionnaires were returned (232 completed, 11 declining to participate). Most researchers recognise the importance of and appear committed to research dissemination. However, most dissemination activity beyond the publishing of academic papers appears to be undertaken an ad hoc fashion. There is some evidence that access to dissemination advice and support may facilitate more policy interactions; though access to such resources is lacking at an institutional level, and advice from funders can be variable. Although a minority of respondents routinely record details about the impact of their research, when asked about impact in relation to specific research projects most were able to provide simple narrative descriptions.Conclusions: Researchers recognise the importance of and appear committed to disseminating the findings of their work. Although researchers are focussed on academic publication, a range of dissemination activities are being applied albeit in an ad hoc fashion. However, what constitutes effective dissemination (in terms of impact and return on investment) remains unclear. Researchers need greater and clearer guidance on how best to plan, resource, and facilitate their dissemination activities

The ‘dark side’ of knowledge brokering

Deploying knowledge brokers to bridge the ‘gap’ between researchers and practitioners continues to be seen as an unquestionable enabler of evidence-based practice and is often endorsed uncritically. We explore the ‘dark side’ of knowledge brokering, reflecting on its inherent challenges which we categorize as: (1) tensions between different aspects of brokering; (2) tensions between different types and sources of knowledge; and (3) tensions resulting from the ‘in-between’ position of brokers. As a result of these tensions, individual brokers may struggle to maintain their fragile and ambiguous intermediary position, and some of the knowledge may be lost in the ‘in-between world’, whereby research evidence is transferred to research users without being mobilized in their day-to-day practice. To be effective, brokering requires an amalgamation of several types of knowledge and a multidimensional skill set that needs to be sustained over time. If we want to maximize the impact of research on policy and practice, we should move from deploying individual ‘brokers’ to embracing the collective process of ‘brokering’ supported at the organizational and policy levels

Community capacity to acquire, assess, adapt, and apply research evidence: a survey of Ontario's HIV/AIDS sector

<p>Abstract</p> <p>Background</p> <p>Community-based organizations (CBOs) are important stakeholders in health systems and are increasingly called upon to use research evidence to inform their advocacy, program planning, and service delivery. To better support CBOs to find and use research evidence, we sought to assess the capacity of CBOs in the HIV/AIDS sector to acquire, assess, adapt, and apply research evidence in their work.</p> <p>Methods</p> <p>We invited executive directors of HIV/AIDS CBOs in Ontario, Canada (n = 51) to complete the Canadian Health Services Research Foundation's "Is Research Working for You?" survey.</p> <p>Findings</p> <p>Based on responses from 25 organizations that collectively provide services to approximately 32,000 clients per year with 290 full-time equivalent staff, we found organizational capacity to acquire, assess, adapt, and apply research evidence to be low. CBO strengths include supporting a culture that rewards flexibility and quality improvement, exchanging information within their organization, and ensuring that their decision-making processes have a place for research. However, CBO Executive Directors indicated that they lacked the skills, time, resources, incentives, and links with experts to acquire research, assess its quality and reliability, and summarize it in a user-friendly way.</p> <p>Conclusion</p> <p>Given the limited capacity to find and use research evidence, we recommend a capacity-building strategy for HIV/AIDS CBOs that focuses on providing the tools, resources, and skills needed to more consistently acquire, assess, adapt, and apply research evidence. Such a strategy may be appropriate in other sectors and jurisdictions as well given that CBO Executive Directors in the HIV/AIDS sector in Ontario report low capacity despite being in the enviable position of having stable government infrastructure in place to support them, benefiting from long-standing investment in capacity building, and being part of an active provincial network. CBOs in other sectors and jurisdictions that have fewer supports may have comparable or lower capacity. Future research should examine a larger sample of CBO Executive Directors from a range of sectors and jurisdictions.</p